Skip to content

Conversation

joegallo
Copy link
Contributor

@joegallo joegallo commented Oct 15, 2025

Fixes a bug introduced by #105718

With allow_duplicates: false, we're pretty careful to not upcast a scalar into a list unless some kind of change has been made to the value -- that is, if you have a value of "2" and you are ignoring duplicates and you try to append "2" onto it, you still have a value of "2" and not ["2"].

With #105718, there's a new behavioral path, that is ignore_empty_values: true. We need to take the same care here that we don't upcast a scalar (or the absence of a value) into a list unless there's actually some value to be appended. That is, if you have ignore_empty_values: true and all the values that you want to append are actually empty, then the resulting operation should be a no-op, but it wasn't! This PR fixes that.

ALSO, I'm super pleased with how innerAppendValues and innerAppendValuesWithDuplicates drop out because of this, I've never been especially happy with those methods existing. 😄

Note: Given that there's another BC, I actually expect this to land in v9.2.0 (I imagine the labels will get corrected at some point).

Additional note: related to #104813, with was the feature request that asked for something like ignore_empty_values to begin with.

@joegallo joegallo requested a review from masseyke October 15, 2025 19:51
@joegallo joegallo added >bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team v9.2.1 v9.3.0 auto-backport Automatically create backport pull requests when merged labels Oct 15, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine
Copy link
Collaborator

Hi @joegallo, I've created a changelog YAML for you.

@joegallo joegallo merged commit 054658d into elastic:main Oct 15, 2025
34 checks passed
@joegallo joegallo deleted the fix-append-edge-case-bug branch October 15, 2025 21:44
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
9.2 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 136649

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending >bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team v9.2.1 v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants